Accelerated data removal in hierarchical storage environments

ABSTRACT

A computer-implemented method for maintaining a storage volume in a virtual tape system includes writing one or more logical volumes associated with a first category and one or more logical volumes associated with a second category to a primary storage in a virtual tape system. The computer-implemented method further includes performing a first automatic removal process in order to free up space on the primary storage, wherein the first automatic removal process removes logical volumes associated with the first category in priority to logical volumes associated with the second category. The computer-implemented method further includes performing a second automatic removal process, wherein the second automatic removal process dynamically alters the priority of the first automatic removal process such that one or more virtual volumes associated with the second category are removed in priority to one or more virtual volumes associated with the first category.

BACKGROUND

The present invention relates generally to the field of hierarchicalstorage management, and more particularly to hierarchical storagemanagement of virtual tape systems.

Hierarchical storage management is a data storage process that movesdata within a tiered storage environment. In a tiered storageenvironment, at least two types of data storage media are delineated bydifferences in attributes, such as price, performance, capacity, andfunction. Accordingly, whether data is stored in one tier or another isdefined by the requirements of the data to be stored.

The use of hierarchical storage management allows an enterprise toreduce the cost of data storage, as well as simplify the retrieval ofdata from slower storage media. Typically, hierarchical storagemanagement is used for deep archival storage of data that is required tobe maintained for a prolonged period at low cost. The need forhierarchical storage management stems from the fact that high-speedstorage devices (e.g., solid state drive arrays) are more expensive (perbyte stored) than slower speed storage devices (e.g., hard disk drives,optical discs, and magnetic tape drives). With hierarchical storagemanagement, infrequently used data files stored on high-speed storagemedia are migrated to slower speed storage media if the data files arenot used (i.e., accessed) for a certain period. When access to the datafiles is required, data is copied from the slower speed storage tofaster disk drives. In effect, hierarchical storage management turns afast disk drive into a cache for the slower mass storage devices.

A virtual tape system (“VTS”) is a cloud or virtual data storage andbackup system that uses magnetic-tape-based consolidated storageinfrastructure to store and retrieve data. A VTS functions like atypical magnetic tape storage system but is enabled and integrated withcloud storage and virtualization techniques. Typically, a VTS isimplemented to achieve enhanced storage management performance whilereducing disk cartridge waste. A VTS eliminates data retrieval latencyby storing an instance of frequently used files in disk caches toprovide faster access to the data. Moreover, a VTS uses a virtual tapelibrary (“VTL”) built over storage virtualization to reduce the amountof tape disk required.

SUMMARY

According to one embodiment of the present invention, acomputer-implemented method for maintaining a storage volume in avirtual tape system is disclosed. The computer-implemented methodincludes writing one or more logical volumes associated with a firstcategory and one or more logical volumes associated with a secondcategory to the primary storage in the virtual tape system. Thecomputer-implemented method further includes performing a firstautomatic removal process in order to free up space on the primarystorage, wherein the first automatic removal process removes logicalvolumes associated with the first category in priority to logicalvolumes associated with the second category. The computer-implementedmethod further includes performing a second automatic removal process,wherein the second automatic removal process dynamically alters thepriority of the first automatic removal process such that one or morevirtual volumes associated with the second category are removed inpriority to one or more virtual volumes associated with the firstcategory.

According to another embodiment of the present invention, a computerprogram product for maintaining a storage volume in a virtual tapesystem is disclosed. The computer program product includes one or morecomputer readable storage media and program instructions stored on theone or more computer readable storage media. The program instructionsinclude instructions to write one or more logical volumes associatedwith a first category and one or more logical volumes associated with asecond category to the primary storage in the virtual tape system. Theprogram instructions further include instructions to perform a firstautomatic removal process in order to free up space on the primarystorage, wherein the first automatic removal process removes logicalvolumes associated with the first category in priority to logicalvolumes associated with the second category. The program instructionsfurther include instructions to perform a second automatic removalprocess, wherein the second automatic removal process dynamically altersthe priority of the first automatic removal process such that one ormore virtual volumes associated with the second category are removed inpriority to one or more virtual volumes associated with the firstcategory.

According to another embodiment of the present invention, a computersystem for maintaining a storage volume in a virtual tape system isdisclosed. The computer system includes one or more computer systemincludes one or more computer processors, one or more computer readablestorage media, and program instructions stored on the computer readablestorage media for execution by at least one of the one or moreprocessors. The program instructions include instructions to write oneor more logical volumes associated with a first category and one or morelogical volumes associated with a second category to the primary storagein the virtual tape system. The program instructions further includeinstructions to perform a first automatic removal process in order tofree up space on the primary storage, wherein the first automaticremoval process removes logical volumes associated with the firstcategory in priority to logical volumes associated with the secondcategory. The program instructions further include instructions toperform a second automatic removal process, wherein the second automaticremoval process dynamically alters the priority of the first automaticremoval process such that one or more virtual volumes associated withthe second category are removed in priority to one or more virtualvolumes associated with the first category.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is an exemplary disk cache in a storage array of a virtual tapelibrary in accordance with at least one embodiment of the invention.

FIG. 1B is an exemplary disk cache in a storage array of a virtual tapelibrary after undergoing a first logical volume auto-removal processover a period of time in accordance with at least one embodiment of theinvention.

FIG. 2 is a functional block diagram of a network computing environment,generally designated 200, suitable for operation of a data removalprogram 201 in accordance with at least one embodiment of the invention.

FIG. 3 is a flow chart diagram depicting operational steps for a dataremoval program 201 in accordance with at least one embodiment of theinvention.

FIG. 4 is a block diagram depicting components of a computer, generallydesignated 400, suitable for executing data removal program 201 inaccordance with at least one embodiment of the invention.

DETAILED DESCRIPTION

Generally, a VTS includes a system (e.g., a special storage device,group of devices, etc.) and software, firmware, etc., that hashierarchical storage management functionality, whereby data is migratedbetween tiered storage. For example, a VTS migrates data between aprimary, high-speed storage media (e.g., hard disk drives and/orsolid-state disk drives) and a secondary, slower speed storage media(e.g., magnetic tape drive). The primary storage media (typically thefaster storage device) is also known as a tape volume cache (“TVC”).

A virtual tape library (“VTL”) is a data storage virtualizationtechnology that utilizes a storage component (usually hard disk storage)as tape libraries or tape drives for use with existing backup software.Virtualizing the disk storage as tape allows integration of VTLs withexisting backup software and existing backup and recovery processes andpolicies. In essence, a VTL writes data in such a way that the dataappears to be stored entirely on tape cartridges when the data isactually located on faster storage devices, such as solid-state drivesand hard disk drives. A VTL can be thought of as a disk array thatmimics tape. To servers, the VTL appears to be a tape library and thedata backup software that writes to it and manages it as if it weretape. However, because a VTL is actually disk-based, it is much fasterthan tape to read and write. The benefits of such virtualization includestorage consolidation and faster data restore processes. Furthermore, bybacking up data to disks instead of tapes, VTL often increasesperformance of both backup and recovery operations.

A VTL stores data on disk drives as logical volumes (i.e., “virtualvolumes”). In some instances, logical volumes are classified and managedaccording to categories of data. Such categories may include but are notlimited to a “scratch” category and a “private” category. A scratchcategory is a category for registering logical volumes that are allowedto be overwritten. Thus, when writing new data to a VTL, the writer ofthe data performs a scratch mount. Upon receiving a scratch mountinstruction, the VTL selects one of the volumes in the scratch categoryand performs the mount. In other words, the VTL selects a volume thatcan be overwritten.

The private category is a category for registering logical volumes thatare not allowed to be overwritten and thus, are to be retained. Whetheror not a logical volume is designated at a “scratch” or a “private”category can be designated by a user or system administrator.Additionally, a user or system administrator may change the designation,such that a logical volume with a “scratch” category can be moved to a“private” category and vice versa. Accordingly, a logical volumedesignated in the “private” category that is no longer needed canultimately be moved from the “private” category to the “scratch”category.

Turning now to the Figures, and particularly to FIG. 1A, an exemplarydisk cache 100A of a disk in a storage array of a VTL can be seen. Asdepicted in FIG. 1A, disk cache 100A includes a first category 110A ofstored data (“scratch” logical volumes), a second category 120A ofstored data (“private” logical volumes), and free space 130A. Firstcategory 110A of stored data includes scratch data 110A₁, scratch data110A₂, scratch data 110A₃, and scratch data 110A₄. Second category 120Aof stored data includes private data 120A₁, private data 121A₂, andprivate data 122A₃. As data is written to disk cache 100A, free space130A decreases.

Once free space 130A on disk cache 100A is equal to and/or below apredetermined threshold value, the VTL is triggered to perform logicalvolume auto-removal processing in an attempt to increase free space 130Aon disk cache 100A. According to typical logical volume auto-removalprocesses, logical volumes in the scratch category are first removed indescending order of capacity (i.e., size) so that free space can beprovided faster. Thus, the order of removal (in descending order ofcapacity) of logical volumes in the scratch category of disk cache 100Awould be scratch data 110A₁, scratch data 110A₂, scratch data 110A₃, andlastly scratch data 110A₄. If all of the logical volumes in the scratchcategory have been removed, then logical volumes in the private categoryare removed in descending order of elapsed time of prior access.

FIG. 1B illustrates an exemplary disk cache 100B of a disk in a storagearray of a VTL after undergoing a first type of logical volumeauto-removal process over a period of time. It should be noted that diskcache 100A is identical to disk cache 100B. However, a portion of thedata written to disk cache 100B is different than the data written todisk cache 100A as a result of logical volume auto-removal processing.

As depicted in FIG. 1B, disk cache 100B includes first category 110B ofstored data (logical volume scratch data), a second category 120B ofstored data (logical volume private data), and free space 130B. Althoughscratch data 110A₁, scratch data 110A₂, scratch data 110A₃, and lastlyscratch data 110A₄ have been removed from disk cache 100A (depicted inFIG. 1A), and thus are no longer stored on disk cache 100B, as scratchdata is removed from disk cache 100A during the logical volumeauto-removal process, new scratch data, as well as new private data isalso being written to disk cache 100A. Accordingly, first category 110Bof stored data on disk cache 100B now includes scratch data 110B₅₇,110B₅₈, and 110B₅₉ . . . 110B_(n). It should be noted that initially, asthe larger capacity logical volumes in the scratch category are removed,free space on the disk cache increases. This is evinced by the fact thatafter undergoing logical volume auto-removal processing, second category120B of stored data of disk cache 100B now includes private data 120B₁,private data 120B₂, private data 120B₃, private data 120B₄, and privatedata 120B₅, wherein private data 120B₁, private data 120B₂, private data120B₃ is the same data as private data 120A₁, private data 120A₂,private data 120A₃ of disk cache 100A (depicted in FIG. 1A).

However, with increasing demand for faster, more powerful and moreefficient ways to store information, optimization of storagetechnologies remains a key challenge in VTSs. Embodiments of the presentinvention recognize several deficiencies with the first type of logicalvolume auto-removal process. Although logical volumes in the scratchcategory are removed in descending order of logical volume capacity (andthereby providing cache free disk space more quickly), the time requiredfor removal of logical volumes is fixed, regardless of a logicalvolume's capacity.

As illustrated by FIGS. 1A and 1B, as logical volumes in the scratchcategory are removed in descending order of capacity (i.e., the largercapacity logical volumes are removed first), logical volumes with asmaller capacity remain on the disk cache. This stems from the fact thatnew, larger capacity logical volumes will continue to be mounted to thedisk cache during the logical volume auto-removal process. As evinced bythe transformation of the initial data stored on disk cache 100A of FIG.1A at the start of the logical volume auto-removal process and the finaldata stored on disk cache 100B after a period of time, the number oflogical volumes in the scratch category on disk cache 100B has actuallyincreased.

Embodiments of the present invention recognize that since the timerequired for removal of a logical volume is fixed, regardless of thelogical volume's capacity, the amount of cache free disk space that canbe freed up decreases as the size of the logical volume being removeddecreases. As the number of small capacity logical volumes loaded on thedisk cache increases, the amount of cache free disk space created by theremoval of such small capacity logical volumes becomes negligible.Accordingly, the amount of cache free disk space that can be freed upper unit of time (i.e., the rate of removal) through typical logicalvolume removal processes ultimately decreases over time.

If the rate of removal continues to be slower than the rate at whichdata is written, the disk will eventually run out of cache free diskspace. The decrease in the rate of removal due to an increase in smallcapacity logical volumes in the scratch category becomes increasinglyproblematic if the disk runs out of cache free disk space. At thispoint, data writing to the VTS can no longer be performed andconsequently, host jobs are suspended.

Embodiments of the present invention provide one or more of: features,characteristics, operations and/or advantages to the first type oflogical volume auto-removal process and generally encompass (i) animprovement to at least the field of hierarchical storage management and(ii) a technical solution to one or more of challenges in the field ofhierarchical storage management. Such challenges in the field ofhierarchical storage management may include, but are not limited to, oneor more of: (i) limitations in the amount of cache free disk space thatcan be freed up per unit of time, (ii) limitations in the removal oflogical volumes in a private category, (iii) limitations in the dataremoval rate from a disk cache, and (iv) shifting respective priority ofdata removal in various logical volumes. Certain embodiments of thepresent invention both recognize and address other challenges that arenot specifically addressed herein but are readily understood to beencompassed by the technical solutions described herein.

Embodiments of the present invention may increase the data removal ratefrom a disk cache (and thereby increase the amount of cache free diskspace that can be freed up per unit of time) by removing logical volumesin the private category. Embodiments of the present invention increasethe data removal rate from a disk cache by dynamically (i.e.,automatically) shifting priority of data removal from logical volumes inthe “scratch category” to logical volumes in the “private category”during a logical volume auto-removal process. In an embodiment, inresponse to shifting the priority of data removal, logical volumes inthe private category are removed in descending order of elapsed time ofprior access. In an embodiment, in response to shifting the priority ofdata removal, logical volumes in the private category are removed indescending order of capacity.

In embodiments of the invention, the priority of data removal isdynamically shifted based, at least in part, on one or more of thefollowing factors: (i) a rate at which data is being written to the diskcache, (ii) a rate of removal (bytes per unit time) of data from thescratch category, and (iii) an amount of free space on the disk cache.In an embodiment, priority of data removal is dynamically shifted fromlogical volumes in the scratch category to logical volumes in theprivate category in response to determining that a rate of removal oflogical volumes from the scratch category is below and/or equal to apredetermined threshold value. In a further embodiment, priority of dataremoval is dynamically shifted from logical volumes in the scratchcategory to logical volumes in the private category in response todetermining that an amount of free space on a disk cache falls below apredetermined threshold value. In yet another embodiment, priority ofdata removal is dynamically shifted from logical volumes in the scratchcategory to logical volumes in the private category in response todetermining that both the rate of removal of logical volumes in thescratch category and the amount of free space on a disk cache has fallenbelow a predetermined threshold value.

Similarly, embodiments of the present invention provide for dynamicallyshifting priority of data removal from logical volumes in the privatecategory to logical volumes in the scratch category in responsedetermining that an amount of cache free disk space on the TVC is abovethe predetermined threshold value.

In some embodiments of the invention, an increase in the data removalrate from a disk cache is accomplished without entirely shiftingpriority of data removal from logical volumes in the “scratch category”to logical volumes in the “private category.” In these embodiments,priority of data removal from logical volumes in the “scratch category”is changed such that priority of data removal is alternated betweenlogical volumes in the scratch category and logical volumes in theprivate category. In an embodiment, priority of data removal is restoredentirely to the removal of logical volumes in the “scratch category” inresponse to determining that an amount of cache free disk space on theTVC exceeds a predetermined threshold value. In a further embodiment,priority of data removal is restored entirely to the removal of logicalvolumes in the “scratch category” in response to determining that a dataremoval rate exceeds a rate at which data is written to a disk cache.

Embodiments of the present invention recognize that although removinglogical volumes in the private category may result in a decrease in theperformance (due to cache misses) of host systems, the removal oflogical volumes in the private category only occurs for a brief periodof time (removal of even a single “private” logical volume will likelyfree up a large amount of space on the disk cache). Moreover,embodiments of the present invention generate a large amount of freespace quickly when the amount of free space on a disk cache becomesscare, thereby averting a worst-case scenario where the disk cachebecomes completely full. At this point, read/write operations cannot beperformed, and application operations are suspended. Thus, embodimentsof the present invention provide a technical solution that generates anet gain in overall performance for the computing system; i.e., theoverall amount of computing resources that are consumed over time isdecreased when compared to alternative solutions. Such computingresources may include but are not limited to central processor usage,volume of storage, network capacity, and the like. Those having skill inthe art will recognize that embodiments of the present invention mayreduce the strain, i.e. resource consumption, experienced by one or bothof hierarchical storage systems as well as hierarchical storage managerswhen compared to alternative solutions.

Referring now to various embodiments of the invention in more detail,FIG. 2 is a functional block diagram of a network computing environment,generally designated 200, suitable for operation of a data removalprogram 201 in accordance with at least one embodiment of the invention.FIG. 2 provides only an illustration of one implementation and does notimply any limitations with regard to the environments in which differentembodiments may be implemented. Many modifications to the depictedenvironment may be made by those skilled in the art without departingfrom the scope of the invention as recited by the claims.

Network computing environment 200 includes host device 202, virtual tapelibrary 204, tape library 206, and client device 208 interconnected overnetwork 210 and network 212. In embodiments of the invention, network210 can be a telecommunications network, a local area network (LAN), awide area network (WAN), such as the Internet, or a combination of thethree, and can include wired, wireless, or fiber optic connections. Inembodiments of the invention, network 212 is a storage area network(“SAN”). Network 212 provides block-level network access to storage,such as virtual tape library 204 and tape library 206. Network 210 andnetwork 212 may include one or more wired and/or wireless networks thatare capable of receiving and transmitting data, voice, and/or videosignals, including multimedia signals that include voice, data, andvideo information. In general, network 210 and network 212 may be anycombination of connections and protocols that will supportcommunications between client device 208, host device 202, virtual tapelibrary 204, tape library 206, and other computing devices (not shown)within network computing environment 200.

In various embodiments of the invention, each of host device 202 andvirtual tape library 204 are computing devices that can be a standalonedevice, a management server, a web server, a mobile device, or any otherelectronic device or computing system capable of receiving, sending, andprocessing data. In other embodiments, host device 202 and virtual tapelibrary 204 represent a server computing system utilizing multiplecomputers as a server system, such as in a cloud computing environment.In an embodiment, host device 202 and virtual tape library 204 representa computing system utilizing clustered computers and components (e.g.database server computers, application server computers, web servercomputers, etc.) that act as a single pool of seamless resources whenaccessed within network computing environment 200. In general, hostdevice 202 and virtual tape library 204 represent any programmableelectronic device or combination of programmable electronic devicescapable of executing machine readable program instructions andcommunicating with client device 208 and tape library 206 within networkcomputing environment 200 via a network, such as network 210 and network212.

Virtual tape library 204 is a data storage system that includes aprimary storage that acts as a cache for a secondary storage andvirtualization software that presents a storage component (e.g., harddisk storage) as tape libraries or tape drives. Virtualizing diskstorage as tape allows integration of virtual tape library 204 withexisting backup software and existing backup and recovery processes andpolicies. For example, virtual tape library 204 may be an IBM TS7720Virtualization Engine (TS7720 VE™) or an IBM 7740 Virtualization Engine(TS7740™). While reference is made to IBM-specific hardware and/orsoftware components, it should be understood that aspects of the presentinvention may be applied equally to other virtual tape librarytechnologies. In embodiments of the invention, virtual tape library 204includes storage array 214 (i.e., disk array). For example, storagearray 214 is a RAID (Redundant Array of Independent Disks) based storagesystem. In some embodiments, storage array 214 is composed of spinninghard disk drives (“HDDs”). In other embodiments, storage array 214 iscomposed of solid-state disk drives. Each disk in storage array 214includes a disk cache, such as disk cache 216.

In various embodiments of the invention, virtual tape library 204receives write operations for data initially directed to be stored on atape drive, such as a tape drive of tape library 206. However, ratherthan directly writing the data to a tape drive of tape library 206,virtual tape library 204 writes (i.e., “saves” or “stores”) the data asa logical volume (i.e., virtual volume) on disk cache 216 of storagearray 214. The data may remain on disk cache 216 until removal of thedata is required, at which point, the data is written to a tape drive oftape library 206. For example, data is removed from a disk cache andwritten to a tape drive based on an amount of free space on the diskcache falling below and/or equaling a predetermined threshold value.

In various embodiments of the invention, virtual tape library 204receives read requests. Upon receiving a read request, virtual tapelibrary 204 determines whether the data is stored on a disk cache ofstorage array 214. If the requested data is stored as a logical volumeon a disk cache of storage array 214, the data is read from the diskcache. However, if the requested data is stored on a tape drive of tapelibrary 206, virtual tape library 204 loads the data from a tapecartridge of tape library 206 to a disk cache of storage array 214, suchthat the data is read from the disk cache.

Virtual tape library 204 includes data removal program 201communicatively coupled to network 210 and network 212. Although dataremoval program 201 is depicted in FIG. 2 as being integrated withvirtual tape library 204, in alternative embodiments, data removalprogram 201 is remotely located from virtual tape library 204. Forexample, data removal program 201 can be integrated with host device202. Virtual tape library may include internal and external hardwarecomponents, as depicted and described in further detail with respect toFIG. 4.

Tape library 206 is an automated tape storage device that includes aplurality of tape drives for writing to and reading from tape media,such as single-reel or two-reel magnetic tape cartridges. For example,tape library 206 may be an IBM TS3400™ Tape Library or an IBM TS3500™Tape Library. While reference is made to IBM-specific hardware and/orsoftware components, it should be understood that aspects of the presentinvention may be applied equally to other tape library technologies. Inembodiments of the invention, tape library 206 includes a plurality oftape media stored in banks or groups of storage slots. For example, tapemedia may include, but is not limited to magnetic tape cartridges,magnetic tape cassettes, and optical tape cartridges. Tape library 206further includes one or more tape drives, a plurality of slots to holdtape cartridges, a barcode reader to identify tape cartridges and anautomated method (e.g., a robot) for loading tapes.

Client device 208 allows a user to access an application running on hostdevice 202 and/or data removal program 201 via a network, such asnetwork 210 and network 212. Client device 208 may be a laptop computer,tablet computer, netbook computer, personal computer (PC), a desktopcomputer, a personal digital assistant (PDA), a smart phone, or anyprogrammable electronic device capable of receiving, sending, andprocessing data. In general, client device 208 represents anyprogrammable electronic device or combination of programmable electronicdevices capable of executing machine readable program instructions andcommunicating with host device 202, virtual tape library 204, tapelibrary 206, and other computing devices (not shown) within computingenvironment 200 via a network, such as network 210 and network 212.

Client device 208 includes user interface 218. User interface 218provides an interface between client device 208, host device 202,virtual tape library 204, and tape library 206. In some embodiments,user interface 218 may be a graphical user interface (GUI) or a web userinterface (WUI) and can display text, documents, web browser windows,user options, application interfaces, and instructions for operation,and includes the information (such a graphic, text, and sound) that aprogram presents to a user and the control sequences the user employs tocontrol the program. In other embodiments, user interface 218 may alsobe mobile application software that provides an interface between clientdevice 208, host device 202, virtual tape library 204, and tape library206.

FIG. 3 is a flow chart diagram depicting operational steps for dataremoval program 201 in accordance with at least one embodiment of theinvention. FIG. 3 provides only an illustration of one implementationand does not imply any limitations with regard to the environments inwhich different embodiments may be implemented. Many modifications tothe depicted environment may be made by those skilled in the art withoutdeparting from the scope of the invention as recited by the claims.

At step S300, data removal program 201 monitors one or more performanceconditions of a disk cache, such as disk cache 216 of a storage array,such as storage array 214. In some embodiments, monitoring a performancecondition includes determining whether a performance condition fallsbelow a predetermined threshold value. In other embodiments, monitoringa performance condition includes determining whether a performancecondition exceeds a predetermined threshold value.

In embodiments of the invention, monitoring a performance conditionincludes determining an amount of free space (i.e., bytes) on disk cache216 of storage array 214. The amount of free space is relative to thesize of the disk cache, which typically ranges from 128 MB in standarddisks to 1 GB in solid state disks.

In embodiments of the invention, monitoring a performance conditionincludes determining a removal rate of logical volumes from disk cache216. In an embodiment, the removal rate of logical volumes is relativeto the number of logical volumes that can be removed per unit of time(e.g., 5 logical volumes per second or 250 logical volumes per minute).In an embodiment, the removal rate of logical volumes is relative to anamount of data that can be removed per unit of time (e.g., 50 MB persecond or 2,500 MB per minute).

In embodiments of the invention, monitoring a performance conditionincludes determining a rate at which data is written (i.e., write speed)to disk cache 216. In some embodiments, the write speed is relative tothe amount of data being written to a disk cache per unit of time (e.g.,150 MB per second). In other embodiments, the write speed is relative tothe number of logical volumes being written to a disk cache per unit oftime (e.g., 3 logical volumes per second). In an embodiment, the writespeed is relative to a number of “private” logical volumes being writtento a disk cache per unit of time. In an embodiment, the write speed isrelative to a number of “scratch” logical volumes being written to adisk cache per unit of time. In an embodiment, the write speed isrelative to a proportion of the number of “private” logical volumes andthe number of “scratch” logical volumes being written to a disk cacheper unit of time.

At decision step S301, data removal program 201 determines whether anamount of free space (i.e., storage space) on disk cache 216 is belowand/or equal to a first predetermined threshold value (e.g., 25% freespace). If an amount of free space on disk cache 216 is above the firstpredetermined threshold value, data removal program 101 returns to stepS300 (decision step “NO” branch). If the amount of free space on diskcache 216 is below and/or equal to the first predetermined thresholdvalue, data removal program 201 proceeds to step S302 (decision step“YES” branch).

At step S302, data removal program 201 performs a first automaticremoval process for disk cache 216. Under the first automatic removalprocess, data removal program 201 removes logical volumes in the scratchcategory in descending order of capacity (i.e., size). If all of thelogical volumes in the scratch category are removed, data removalprogram 201 removes logical volumes in the private category indescending order of elapsed time of prior access. For example, a“private” logical volume accessed 20 minutes ago would be removed beforea “private” logical volume accessed only 5 minutes ago.

At decision step S303, data removal program 201 determines whether theamount of free space (i.e., storage space) on disk cache 216 is belowand/or equal to a second predetermined threshold value (e.g., 15% freespace). It should be noted that the first threshold value in decisionstep S301 and the second threshold value in decision step S303 may beany predetermined numerical value, so long as the first threshold valueis greater than the second threshold value. For example, the firstthreshold value corresponds to 20% free space on disk cache 216 and thesecond threshold value corresponds to 10% free space on disk cache 216.

In embodiments of the invention, decision step S303 is invoked duringthe first automatic removal process of step S302. For example, theamount of data being written to the disk cache 216 of storage array 214may exceed the rate of removal of data in accordance with the firstautomatic removal process. Thus, if the amount of data being written todisk cache 216 exceeds the amount data being removed from disk cache216, the amount of free space on disk cache 216 can continue to decreaseeven though the first automatic removal process is occurring.

If the amount of free space on disk cache 216 is above the secondpredetermined threshold value, data removal program 201 returns to stepS301 (decision step “NO” branch). If the amount of free space on thedisk cache is below and/or equal to the second predetermined thresholdvalue, data removal program 201 proceeds to step S304 (decision step“YES” branch).

At step S304, data removal program 201 predicts a removal rate per unitof time (e.g., bytes of data per second) for one or more “scratch”logical volumes stored on disk cache 216 of storage array 214. In someembodiments, a first predicated removal rate is determined for thelargest capacity “scratch” logical volume in accordance with the firstautomatic removal process (“scratch” logical volumes are removed indescending order of capacity). In these embodiments, the firstpredicated removal rate is based, at least in part, on: (i) a number oflogical volumes that can be removed per unit of time and (ii) a capacityof the next logical volume slotted for removal.

In other embodiments, a second predicted removal rate is determined fortwo or more “scratch” logical volumes in accordance with the firstautomatic removal process (“scratch” logical volumes are removed indescending order of capacity). Thus, if the predicted removal rate isdetermined for three “scratch” logical volumes, the first logical volumewould have the largest capacity, followed by the second and thirdlogical volumes, respectively. In these embodiments, the secondpredicted removal rate is based, at least in part, on: (i) a number oflogical volumes that can be removed per unit of time and (ii) an averagecapacity of two or more logical volumes slotted for removal.

In embodiments of the invention, whether data removal program 201utilizes the first predicted removal rate or the second predictedremoval rate is based, at least in part, on a length of time since thefirst automatic removal process was invoked. This is based on theassumption that the length of time that data has been removed from adisk cache will dictate a degree of similarity between those “scratch”logical volumes that remain on the disk cache. As the length of timeincreases, the degree of similarity in capacity size between theremaining “scratch” logical volumes stored on the data cache increases.This is based on the fact that data removal program 201 will continue toremove “scratch” logical volumes in descending order of capacity. Thus,after undergoing the first automatic removal process for a longer periodof time, the “scratch” logical volumes that have a larger capacity willalready have been removed, leaving the “scratch” logical volumes thathave a smaller capacity on disk cache 216.

On the other hand, as the length of time decreases, the degree ofsimilarity in capacity size between the remaining “scratch” logicalvolumes stored on disk cache 216 decreases. This is based on the factthat data removal program 201 will continue to remove “scratch” logicalvolumes in descending order of capacity. Thus, after undergoing thefirst automatic removal process for a shorter period of time, a mixtureof “scratch” logical volumes with various capacities are likely toremain on disk cache 216.

In any of these embodiments, data removal program 201 analyzeshistorical automatic removal processes to determine a degree ofsimilarity in capacity size of remaining “scratch” logical volumesstored on disk cache 216. For example, data removal program 201 comparesthe length of time since the first automated removal process commencedto a previous automatic process invoked for a similar length of time todetermine the degree of similarity between “scratch” logical volumes.

At decision step S305, data removal program 201 determines whether therate at which data is being written to disk cache 216 is less than orequal to a predicted removal rate per unit time. In some embodiments,the determination is based on the first predicted removal rate. In otherembodiments, the determination is based on the second predicted removalrate. If the rate at which data is being written to disk cache 216 isless than or equal to a predicated removal rate, data removal program201 returns to step S301 (decision step “YES” branch). If the rate atwhich data is being written to disk cache 216 is greater than apredicated removal rate, data removal program 201 proceeds to step S306.

At step S306, data removal program 201 performs a second automaticremoval process. Under the second automatic removal process, dataremoval program 201 dynamically changes the priority of data removalfrom “scratch” logical volumes in accordance with the first automaticremoval process to “private” logical volumes. In an embodiment, dataremoval program 201 removes logical volumes in the “private” category indescending order of capacity (i.e., size). In an embodiment, dataremoval program 201 removes logical volumes in the “private category” indescending order of elapsed time of prior access. In an embodiment, dataremoval program 201 alternates between removing logical volumes indescending order of capacity and logical volumes in descending order ofelapsed time of prior access.

In some embodiments, removal of logical volumes in the “private”category in accordance with the second automatic removal process isbased, at least in part, on whether a “private” logical volume stored ona disk cache virtual tape library 204 has been replicated on a diskcache of a second, distinct virtual tape library. It should be notedthat cache misses may occur due to the removal of “private” logicalvolumes from the disk cache. Accordingly, if a cache miss occurs, thespeed at which a “private” logical volume can be re-written to diskcache 216 of virtual tape library 204 is crucial to the performance ofhost device 102. Since a logical volume can be obtained from anotherremote virtual tape library faster than from a tape library, such astape library 206, data removal program 201 may remove a “private”logical volume that is duplicated on another virtual tape library priorto removing a “private” logical volume that has not been duplicated onanother virtual tape library.

At decision step S307, data removal program 201 determines whether theamount of free space on disk cache 216 is below and/or equal to thesecond predetermined threshold value. In some embodiments, data removalprogram 201 determines the amount of free space on disk cache 216 aftereach “private” logical volume is removed in accordance with the firstpredicted removal rate of the second automatic removal process. In otherembodiments, data removal program 201 determines the amount of freespace on disk cache 216 after two or more “private” logical volumes areremoved in accordance with the second predicted removal rate of thesecond automatic removal process. If the amount of free space on diskcache 216 is above the second predetermined threshold level (decisionstep “NO” branch), data removal program 201 returns to decision stepS301. If the amount of free space on disk cache 216 is below and/orequal to the second predetermined threshold level, data removal program201 returns to step S306.

FIG. 4 is a block diagram depicting components of a computer 400suitable for executing data removal program 201 in accordance with atleast one embodiment of the invention. FIG. 4 displays the computer 400,one or more processor(s) 404 (including one or more computerprocessors), a communications fabric 402, a memory 406 including, a RAM416, and a cache 418, a persistent storage 408, a communications unit412, I/O interface(s) 414, a display 422, and external device(s) 420. Itshould be appreciated that FIG. 4 provides only an illustration of oneembodiment and does not imply any limitations with regard to theenvironments in which different embodiments may be implemented. Manymodifications to the depicted environment may be made.

As depicted, the computer 400 operates over the communications fabric402, which provides communications between the computer processor(s)404, memory 406, persistent storage 408, communications unit 412, andinput/output (I/O) interface(s) 414. The communications fabric 402 maybe implemented with any architecture suitable for passing data orcontrol information between the processor(s) 404 (e.g., microprocessors,communications processors, and network processors), the memory 406, theexternal device(s) 420, and any other hardware components within asystem. For example, the communications fabric 402 may be implementedwith one or more buses.

The memory 406 and persistent storage 408 are computer readable storagemedia. In the depicted embodiment, the memory 406 comprises a randomaccess memory (RAM) 416 and a cache 418. In general, the memory 406 maycomprise any suitable volatile or non-volatile one or more computerreadable storage media.

Program instructions for data removal program 201 be stored in thepersistent storage 408, or more generally, any computer readable storagemedia, for execution by one or more of the respective computerprocessor(s) 404 via one or more memories of the memory 406. Thepersistent storage 408 may be a magnetic hard disk drive, a solid statedisk drive, a semiconductor storage device, read-only memory (ROM),electronically erasable programmable read-only memory (EEPROM), flashmemory, or any other computer readable storage media that is capable ofstoring program instructions or digital information.

The media used by the persistent storage 408 may also be removable. Forexample, a removable hard drive may be used for persistent storage 408.Other examples include optical and magnetic disks, thumb drives, andsmart cards that are inserted into a drive for transfer onto anothercomputer readable storage medium that is also part of the persistentstorage 408.

The communications unit 412, in these examples, provides forcommunications with other data processing systems or devices. In theseexamples, the communications unit 412 may comprise one or more networkinterface cards. The communications unit 412 may provide communicationsthrough the use of either or both physical and wireless communicationslinks. In the context of some embodiments of the present invention, thesource of the various input data may be physically remote to thecomputer 400 such that the input data may be received, and the outputsimilarly transmitted via the communications unit 412.

The I/O interface(s) 414 allow for input and output of data with otherdevices that may operate in conjunction with the computer 400. Forexample, the I/O interface(s) 414 may provide a connection to theexternal device(s) 420, which may be as a keyboard, keypad, a touchscreen, or other suitable input devices. External device(s) 420 may alsoinclude portable computer readable storage media, for example thumbdrives, portable optical or magnetic disks, and memory cards. Softwareand data used to practice embodiments of the present invention may bestored on such portable computer readable storage media and may beloaded onto the persistent storage 408 via the I/O interface(s) 414. TheI/O interface(s) 414 may similarly connect to a display 422. The display422 provides a mechanism to display data to a user and may be, forexample, a computer monitor.

The present invention may be a system, a method, and/or a computerprogram product at any possible technical detail level of integration.The computer program product may include a computer readable storagemedium (or media) having computer readable program instructions thereonfor causing a processor to carry out aspects of the present invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Smalltalk, C++ or the like, andconventional procedural programming languages, such as the “C”programming language or similar programming languages. The computerreadable program instructions may execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone softwarepackage, partly on the user's computer and partly on a remote computeror entirely on the remote computer or server. In the latter scenario,the remote computer may be connected to the user's computer through anytype of network, including a local area network (LAN) or a wide areanetwork (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider).In some embodiments, electronic circuitry including, for example,programmable logic circuitry, field-programmable gate arrays (FPGA), orprogrammable logic arrays (PLA) may execute the computer readableprogram instructions by utilizing state information of the computerreadable program instructions to personalize the electronic circuitry,in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a readable storage medium that can direct acomputer, a programmable data processing apparatus, and/or other devicesto function in a particular manner, such that the computer readablestorage medium having instructions stored therein comprises an articleof manufacture including instructions which implement aspects of thefunction/act specified in the flowchart and/or block diagram block orblocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof computer program instructions, which comprises one or more executableinstructions for implementing the specified logical function(s). In somealternative implementations, the functions noted in the block may occurout of the order noted in the Figures. For example, two blocks shown insuccession may, in fact, be executed substantially concurrently, or theblocks may sometimes be executed in the reverse order, depending uponthe functionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

The descriptions of the various embodiments of the present inventionhave been presented for purposes of illustration but are not intended tobe exhaustive or limited to the embodiments disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope and spirit of the describedembodiments. The terminology used herein was chosen to best explain theprinciples of the embodiments, the practical application or technicalimprovement over technologies found in the marketplace, or to enableothers of ordinary skill in the art to understand the embodimentsdisclosed herein.

What is claimed is:
 1. A computer-implemented method for maintaining astorage volume in a virtual tape system, the computer-implemented methodcomprising: writing one or more logical volumes associated with a firstcategory and one or more logical volumes associated with a secondcategory to a primary storage in a virtual tape system; performing afirst automatic removal process in order to free up space on the primarystorage, wherein the first automatic removal process removes logicalvolumes associated with the first category in priority to logicalvolumes associated with the second category; and performing a secondautomatic removal process, wherein the second automatic removal processdynamically alters the priority of the first automatic removal processsuch that one or more virtual volumes associated with the secondcategory are removed in priority to one or more virtual volumesassociated with the first category.
 2. The computer-implemented methodof claim 1, wherein: (i) the primary storage in the virtual tape systemis a disk cache of a first virtual tape library and (ii) the secondarystorage device is a tape drive of a tape library.
 3. Thecomputer-implemented method of claim 1, wherein the primary storage actsas a cache for the secondary storage in the virtual tape system.
 4. Thecomputer-implemented method of claim 1, wherein: (i) the first categoryis a scratch category for registering those virtual volumes that includedata that can be overwritten and (ii) the second category is a privatecategory for registering those virtual volumes that include data that isnot allowed to be lost.
 5. The computer-implemented method of claim 1,wherein the first automatic removal process is performed based, at leastin part, on an amount of free space on the primary storage being lessthan or equal to a first predetermined threshold value.
 6. Thecomputer-implemented method of claim 1, wherein the first automaticremoval process comprises: removing logical volumes associated with thefirst category in descending order of capacity; and removing, inresponse to removing all of the logical volumes associated with thefirst category, logical volumes associated with the second category indescending order of prior access.
 7. The computer-implemented method ofclaim 1, wherein performing the second automatic removal process isbased, at least in part, on an amount of free space on the primarystorage being less than or equal to a second predetermined thresholdvalue, the second predetermined threshold value being less than thefirst predetermined threshold value.
 8. The computer-implemented methodof claim 1, wherein performing the second automatic removal process isfurther based, at least in part, on a rate at which data is beingwritten to the primary storage being greater than a predicted removalrate.
 9. The computer-implemented method of claim 8, further comprising:determining a first predicted removal rate based, at least in part, on:(i) a rate of removal of logical volumes associated with the firstcategory in accordance with the first automatic removal process and (ii)a capacity of the largest logical volume slotted for removal; anddetermining a second predicted removal rate based, at least in part, on:(i) a rate of removal of logical volumes associated with the firstcategory in accordance with the first automatic removal process and (ii)an average capacity of the two or more logical volumes slotted forremoval.
 10. The computer-implemented method of claim 9, wherein thefirst predicted rate or the second predicted rate is selected based, atleast in part, on a length of time since the first automatic removalprocess commenced.
 11. The computer-implemented method of claim 1,wherein the second automatic removal process removes logical volumes inthe second category in descending order of capacity.
 12. Thecomputer-implemented method of claim 1, wherein the second automaticremoval process removes logical volumes in the second category indescending order of elapsed time of prior access.
 13. Thecomputer-implemented method of claim 1, wherein the second automaticremoval process alternates between removing logical volumes indescending order of capacity and logical volumes in descending order ofelapsed time of prior access.
 14. The computer-implemented method ofclaim 1, wherein selecting a logical volume for removal in accordancewith the second automatic removal process is based, at least in part, onwhether the logical volume is replicated on a second virtual tapelibrary.
 15. A computer program product for maintaining a storage volumein a virtual tape system, the computer program product comprising one ormore computer readable storage media and program instructions stored onthe one or more computer readable storage media, the programinstructions including instructions to: write one or more logicalvolumes associated with a first category and one or more logical volumesassociated with a second category to the primary storage in the virtualtape system; perform a first automatic removal process in order to freeup space on the primary storage, wherein the first automatic removalprocess removes logical volumes associated with the first category inpriority to logical volumes associated with the second category; andperform a second automatic removal process, wherein the second automaticremoval process dynamically alters the priority of the first automaticremoval process such that one or more virtual volumes associated withthe second category are removed in priority to one or more virtualvolumes associated with the first category.
 16. The computer programproduct of claim 15, wherein: (i) the primary storage in the virtualtape system is a disk cache of a first virtual tape library and (ii) thesecondary storage device is a tape drive of a tape library.
 17. Thecomputer program product of claim 15, wherein the primary storage actsas a cache for the secondary storage in the virtual tape system.
 18. Thecomputer program product of claim 15, wherein: (i) the first category isa scratch category for registering those virtual volumes that includedata that can be overwritten and (ii) the second category is a privatecategory for registering those virtual volumes that include data that isnot allowed to be lost.
 19. The computer program product of claim 15,wherein the instructions to perform the first automatic removal processis based, at least in part, on an amount of free space on the primarystorage being less than or equal to a first predetermined thresholdvalue.
 20. A computer system for accelerated data removal formaintaining a storage volume in a virtual tape system, the computersystem comprising: one or more computer processors; one or more computerreadable storage media; computer program instructions; the computerprogram instructions being stored on the one or more computer readablestorage media for execution by the one or more computer processors; andthe computer program instructions including instructions to: write oneor more logical volumes associated with a first category and one or morelogical volumes associated with a second category to the primary storagein the virtual tape system; perform a first automatic removal process inorder to free up space on the primary storage, wherein the firstautomatic removal process removes logical volumes associated with thefirst category in priority to logical volumes associated with the secondcategory; and perform a second automatic removal process, wherein thesecond automatic removal process dynamically alters the priority of thefirst automatic removal process such that one or more virtual volumesassociated with the second category are removed in priority to one ormore virtual volumes associated with the first category.